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~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 

All claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not included 
herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS 
NOTICE OF ALLOWABILITY IS NOT A GRANT OF PATENT RIGHTS. This application is subject to withdrawal from issue at the initiative 
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International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 

1 . In response to the BPAI decision from 7/296/20 1 0, the previous grounds of rejection 
have been withdrawn. 

Examiner's Amendment 

2. An examiner's amendment to the record appears below. Should the changes and/or 
additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 
1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the 
payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview with 
JohnBurbage on 10/05/10. 
In the Claims: 

Please amend claims 1, 5, 15, 28, 40 and 53 as set forth below. 

1 . A method of adjusting a voice prompt of a system based upon a state of a user of the 
system, the method comprising: 

receiving an utterance of the user; 

obtaining utterance parameters from the utterance, the utterance parameters indicating the 
state of the user; 

determining the state of the user based upon an utterance parameter vector; and 
adjusting the voice prompt by adjusting at least one of a tone of voice of the voice 
prompt, a content of the voice prompt, a prosody of the voice prompt, and a gender of the voice 
prompt based upon the determined state of the user, wherein obtaining utterance parameters 
comprises: partitioning the utterance into segments; 
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assigning one of a plurality of classifications to each segment, each classification 
corresponding to at least one of a plurality of states of the user; and 

determining a total number of one of the plurality of classifications divided by a total 
number of segments of the utterance for each of the plurality of states of the user to create the 
utterance parameter vector. 

5 . The method of claim 1 , wherein converting the utterance parameter vector to an 
indication comprises applying a function to the utterance parameter vector to generate one of a 
scalar, a vector of fuzzy classes, and an index representing the state of the user. 
15. A method of adjusting a voice prompt of a system based upon a state of a user of the 
system, the method comprising: 

receiving an utterance of the user; 

obtaining utterance parameters from the utterance, the utterance parameters indicating the 
state of the user; 

generating an utterance parameter vector based upon the obtained utterance parameters; 
converting the utterance parameter vector to an indication representing the state of the 

user; 

determining the state of the user based upon the indication; and 

adjusting the voice prompt based upon the determined state of the user, wherein obtaining 
utterance parameters comprises: 

partitioning the utterance into segments; and 

assigning one of a plurality of classifications to each segment, each classification 
corresponding to at least one of a plurality of states of the user, and wherein generating the 
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utterance parameter vector comprises: determining the number of segments for each 
classification; and dividing the number of segments for each classification by a total number of 
segments in the utterance. 

28. A system adjusting a voice prompt based upon a state of a user of the system, the system 
comprising: 

a signal processing module for obtaining utterance parameters from utterance received 
from the user, the utterance parameters indicating the state of the user; 

an utterance parameter vector generation module for generating an utterance parameter 
vector based upon the obtained utterance parameters; 

a user state determination module for converting the utterance parameter vector to an 
indication representing the state of the user and determining the state of the user based upon the 
indication; and 

a speech waveform storage module for selecting an audio waveform for the voice prompt 
based upon the determined state of the user, wherein the signal processing module obtains the 
utterance parameters by: 

partitioning the utterance into segments; and 

assigning one of a plurality of classifications to each segment, each classification 
corresponding to at least one of a plurality of states of the user, wherein the utterance parameter 
vector generation module generates the utterance parameter vector by: 

determining the number of segments for each classification; and 

dividing the number of segments for each classification by a total number of segments in 
the utterance. 
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40. A system adjusting a voice prompt based upon a state of a user of the system, the system 
comprising: 

a signal processing module for obtaining utterance parameters from utterance received 
from the user, the utterance parameters indicating the state of the user; 

an utterance parameter vector generation module for generating an utterance parameter 
vector based upon the obtained utterance parameters; 

a user state determination module for converting the utterance parameter vector to an 
indication representing the state of the user and determining the state of the user based upon the 
indication; and 

a speech synthesizer module for generating an audio waveform of the voice prompt based 
upon the determined state of the user, wherein the signal processing module obtains the 
utterance parameters by: 

partitioning the utterance into segments; and 

assigning one of a plurality of classifications to each segment, each classification 
corresponding to at least one of a plurality of states of the user, wherein the utterance parameter 
vector generation module generates the utterance parameter vector by: 

determining the number of segments for each classification; and 

dividing the number of segments for each classification by a total number of segments in 
the utterance. 

53. A system adjusting a voice prompt based upon a state of a user of the system, the system 
comprising: 

means for obtaining utterance parameters from utterance received from the user, the 
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utterance parameters indicating the state of the user; 

means for determining the state of the user based upon the utterance parameters; and 

means for adjusting the voice prompt by adjusting at least one of a tone of voice of the 

voice prompt, a content of the voice prompt, a prosody of the voice prompt, and a gender of 

the voice prompt based upon the determined state of the user, wherein the means for 

obtaining utterance parameters comprises: 

means for partitioning the utterance into segments; and 

means for assigning one of a plurality of classifications to each segment, each 

classification corresponding to at least one of a plurality of states of the user, wherein the 

means for determining the state of the user based upon the utterance comprises: 

means for generating an utterance parameter vector based upon the utterance parameter 

by (1) determining the number of segments for each classification, and (2) dividing the number 

of segments for each classification by a total number of segments in the utterance. 

Cancel claims: 3, 4, 17, 30, and 42. 

Allowable Subject Matter 
3. Claims 1, 5-15, 18-28, 31-40, and 43-61 are allowed. 

The following is an examiner's statement of reasons for allowance: 
The pending claims are directed to a method, system, and means for adjusting a voice 
prompt of a system based upon a state of a user of the system, the method comprising: receiving 
an utterance of the user; obtaining utterance parameters from the utterance, the utterance 
parameters indicating the state of the user; determining the state of the user based upon an 
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utterance parameter vector; and adjusting the voice prompt by adjusting at least one of a tone of 
voice of the voice prompt, a content of the voice prompt, a prosody of the voice prompt, and a 
gender of the voice prompt based upon the determined state of the user, wherein obtaining 
utterance parameters comprises: partitioning the utterance into segments; assigning one of a 
plurality of classifications to each segment, each classification corresponding to at least one of a 
plurality of states of the user; and determining a total number of one of the plurality of 
classifications divided by a total number of segments of the utterance for each of the plurality of 
states of the user to create the utterance parameter vector. 
The closest prior art references are: 

Cooper (US 6,757,362) teaches receiving an utterance of the user (col. 2, line 39); 
obtaining utterance parameters from the utterance, the utterance parameters indicating the state 
of the user (col. 43, line 52 - col. 44, line 5, wherein the user's emotional state is determined 
based on words of the received utterance); determining the state of the user based upon the 
utterance parameters (col. 43, lines 62-67, wherein the Virtual Assistant (VA) determines the 
emotional state of the user based on the received utterances from the user) ; adjusting the voice 
prompt by adjusting at least one of a tone of voice of the voice prompt, a content of the voice 
prompt, a prosody of the voice prompt, and a gender of the voice prompt based upon the 
determined state of the user (col. 44, lines 1-4); and wherein obtaining parameters comprises 
partitioning the utterance into segments (col. 2, lines 58-64, wherein the system identifies the 
boundaries between the spoken words, by detecting a user's pauses when recording a message). 

Nicholson (Emotion Recognition in Speech Using Neural Networks, Neural Computing 
& Applications, Volume 9, Number 4, pages 290-296, 2000) teaches emotion recognition in 
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speech, wherein a received spoken utterance is identified and divided into a plurality of vectors 
representing intervals or segments of equal length in time (paragraph 2.4.2 Speech feature 
extraction on page 292), and then assigning an emotional state to every vector as shown in the 
emotion recognition stage of Fig. 3, i.e. vl (anger), v2 (sadness) . . .v8 (neutral), see pages 292- 
293. 

However, the prior art does not teach or suggests determining the emotional state of a 
user based upon an utterance parameter vector, wherein the utterance parameter vector is created 
by determining a total number of one of a plurality of classifications divided by a total number of 
segments of the utterance for each of a plurality of states of the user. 

Any comments considered necessary by applicant must be submitted no later than the 
payment of the issue fee and, to avoid processing delays, should preferably accompany the issue 
fee. Such submissions should be clearly labeled "Comments on Statement of Reasons for 
Allowance." 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Abdelali Serrou whose telephone number is 571-272-7638. The 
examiner can normally be reached on 8:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, James S. Wozniak can be reached on 571-272-7632. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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Examiner, Art Unit 2626 
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Supervisory Patent Examiner, Art Unit 
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